The present invention relates to auto white balance used in electronics still cameras video cameras.
To reproduce a white subject so as to look white, auto white balance control is provided in video cameras or digital still cameras. Well known as the conventional auto white balance method is the type of controlling the balance of RGB components (the three primary color components of red, green and blue) of each pixel signal in such a manner that the average of the entire image results in an achromatic color. However, this method suffered from a deficiency that improper white balance control may often be made in case chromatic colors occupy most regions of the image.
Such an improper white balance control is called a color failure. Known as an auto white balance control method to relieve this color failure is a technique disclosed in Japanese Patent Laid-open Pub. No. Hei 5-292533. This technique splits an image into a plurality of blocks, figures out an average value of RGB of each block, and extracts only the blocks of which average values belong to a predetermined range. The control of the RGB components is then made so that the RGB average values of the extracted blocks result in a achromatic color.
Another auto white balance method to lessen the color failure is seen in Japanese Patent Laid-open Pub. No. Hei 5-7369. This method puts limitations on the range of values of the white balance control signal to obviate the excessive white balance control.
Such methods were effective insofar as the light sources illuminating the subject were limitative, although they could not provide a good white balance control when unassumed light sources were used for illumination or when plural types of light sources were simultaneously used for illumination.
An improved auto white balance control apparatus in Japanese Patent Laid-open Pub. No. Hei 8-289314. This apparatus splits an image into a plurality of blocks and extracts a block group for which it is judged that a white subject under the fluorescent light was photographed, a block group for which it is judged that a white subject under the daylight or tungsten light was photographed, and a block group having a color close to that of the brightest block in an image. Then, the average values of RGB components are figured out for each block group so that the results are mixed with one another in accordance with a previously defined rule to obtain a value for use as the white balance control signal. This apparatus permits the white balance-signal to reflect the RGB average values of the block group having a color approximate to that of the brightest block in the image, to thereby provide a proper white balance control even under unassumed light sources or plural types of light sources.
Up until now, an apparatus is also well known which has a function to provide a fixed white balance control for the user specified light source, separately from the auto white balance control.
In Japanese Patent Laid-open Pub. No. Hei 8-289314, values of dark blocks were used to determine the average values of the block groups because the above block groups were extracted on the basis of only the colors. The color appearing as an image is influenced by both the color of an object itself and the color of the light source illuminating the object. Herein, there is a strong possibility that the dark blocks may not reflect the color of the light source, so that it may possibly cause an error to allow the dark block information to act on the auto white balance control signal. It was therefore often difficult to achieve a good white balance performance by the above conventional method allowing the dark block information to act on the control signal merely because of an approximation in color to the brightest block.
It is the object of the present invention to provide an auto white balance apparatus capable of achieving a white balance control through the extraction of only blocks which strongly reflect the color of the light source.
An auto white balance apparatus comprising:
block split means for splitting an input image into a plurality of blocks;
representative value calculation means for determining, for each split block, a representative value containing a brightness and a color difference representative of each block on the basis of color values of pixels lying within each block;
maximum brightness block search means for finding a maximum brightness block among said plurality of blocks on the basis of said representative values of said blocks;
brightness threshold value calculation means for determining a brightness threshold value on the basis of the brightness contained in said representative value of each block;
high brightness color block average value calculation means for defining a high brightness region as a region indicative of a color analogous to said representative value of said maximum brightness block in a color space on the basis of said representative value of said maximum brightness block, to extract blocks having representative values contained in said high brightness region as high brightness color blocks from blocks having a higher brightness than said brightness threshold value among said plurality of blocks, to thereby determine an average value of representative values of said high brightness color blocks thus extracted;
light source block average value calculation means provided correspondingly for each of assumed light sources, for extracting blocks having representative values contained in an associated light source region defined as a region indicative of the color of a white object under each associated light source in said color space, as light source blocks corresponding to each associated light source, from blocks having a higher brightness than said brightness threshold value among said plurality of blocks, to thereby determine an average value of representative values of associated said light source blocks thus extracted;
white balance control signal calculation means for determining a white balance control signal on the basis of an average value of representative values of said high brightness color blocks and an average value of representative values of said light source blocks for each light source; and
white balance means for providing a white balance control for each pixel of said input image by use of said white balance control signal acquired by said white balance control signal calculation means.
According to this configuration, the white balance control signal is figured out on the basis of only the values of the blocks having a higher brightness than the brightness threshold value, thereby preventing the values of dark blocks having a strong possibility that the color of the light source may not be reflected from adversely affecting the white balance control signal.
In a preferred aspect of the present invention, the brightness threshold value is determined on the basis of the brightness of the maximum brightness block and the average brightness of all the blocks. This enables the brightness threshold value to vary depending on the situations of the individual images.
In another aspect of the present invention, an average value of representative values of blocks associated with each light source is modified by use of a standard value previously defined for each light source, whereby the value of each average value can come closer to the color of the light from the light source, ensuring an appropriate white balance for the color of the light from the light source.
A further aspect of the present invention further comprises means for detecting the brightness of a subject. The white balance control signal calculation means include second modification means for modifying an average value of representative values of the light source blocks associated with a light source of a type used as an indoor light source, with a weight which becomes smaller accordingly as the subject brightness is higher. The white balance control signal calculation means determine the white balance control signal by use of the modified average value. According to this aspect, it can be prevented to provide a white balance for the indoor light source when the subject brightness is high.
In a further aspect of the present invention, the block representative values are figured out by use of values of pixels except saturated pixels among pixels constituting associated blocks, whereby it is possible to obtain the representative values which are considered to well reflect the color of the light from the light source.
In a further aspect of the present invention, the average value is determined from only representative values of blocks other than blocks containing the number of saturated pixels exceeding a predetermined reference value, whereby it is possible to obtain a white balance control signal which well reflects the color of the light from the light source.
In a further aspect of the present invention, if a difference between the value of a predetermined pixel within a block and the representative value of the block exceeds a predetermined value, the representative value of the block is reduced to a predetermined ratio in the calculation of the average value of the block representative values. According to this aspect, there can be reduced the influence on the white balance control signal, of blocks which belonged to a light source region by chance as a result of the averaging processing.
In a further aspect of the present invention, upon the calculation of the average value of the block representative values, a larger weight is imparted to representative values of blocks closer to the center of a picture, to obtain an average value. This enables an appropriate white balance to be effected on the central portion of the picture which is generally considered to be important.
A further aspect of the present invention further comprises all block average value calculation means for determining an all block average value which is obtained by averaging representative values of all blocks of an input image. The white balance control signal calculation means determine the white balance control signal taking into consideration the all block average value as well. This aspect can prevent the white balance from abruptly varying depending on a slight change of the composition, by taking into consideration the all block average value which may often be closer to the color of the light from the light source.
In this aspect, if the all block average value does not belong to any light source regions, the all block average value is preferably modified to a predetermined small value, to be used for the calculation of the white balance control signal. In case the all block average value is an inappropriate value which does not reflect the color of the light from the light source, this enables the value to be less contributed to the white balance control signal.
A further aspect of the present invention further comprises means for detecting the brightness of a subject, and daylight block average value calculation means assuming the sun as a light source, the daylight block average value calculation means being one of the light source block average value calculation means. The daylight block average value calculation means reduce the dimensions of the light source region associated with daylight so as to exclude a high color temperature portion if the subject brightness is higher than a predetermined threshold value. The daylight block average value calculation means increases the dimensions of the light source region associated with the daylight so as to contain the high color temperature portion if the subject brightness is lower than the threshold value. According to this aspect, a proper white balance can automatically be applied to both the daytime blue sky scene and shade or other scene, of which color temperatures are close to each other.
In a further aspect of the present invention, upon the strobe light emission, an average value of representative values of the light source blocks assuming the sun as the light source is modified by using a previously defined strobe standard value in lieu of a standard value defined for the daylight, which result is then used for the calculation of the white balance control signal. According to this aspect, it is possible to process both the daylight and the strobe light by a common unit, which contributes to a simple apparatus configuration.
A further aspect of the present invention has a light source specification mode in which the white balance is achieved in view of only a specified light source. The white balance control signal calculation means in the light source specification mode determine the white balance control signal on the basis of only the average value of light source blocks associated with the specified light source. According to this aspect, by determining the white balance control signal on the basis of the average value of the light source blocks corresponding to the specified light source, it is possible to provide a dynamic control of the light source specified white balance depending on the situations of the scenes.
A further aspect of the present invention further comprises judging means for judging whether the illumination environment has changed between the present time and most recent time on the basis of comparison between a white balance signal acquired from an input image and a white balance signal of the most recent image; and
storage means for storing acquired white balance control signals for the duration in which judgment is made that there is no change in the illumination environment. When the judgment means have judged that there is no change in the illumination environment, the white balance control signal calculation means correct the white balance control signal acquired from the input image by the use of the white balance control signals stored in the storage means and then provide it as output to the white balance means. According to this aspect, for the duration in which judgment is made that there is no change in the illumination environment, the current control signal is smoothed by the white balance control signals which have been stored so far. This eliminates any possibility that the white balance may abruptly vary as long as the illumination environment is unvaried.
In a further aspect of the present invention, the white balance control signal calculation means determine, for all previously assumed light sources, an index value indicative of a possibility that each light source is an illumination light source on the basis of each associated light source block count. If the index value is smaller than a predetermined threshold value for the all previously assumed light sources, the white balance control signal calculation means judge that the white balance control signal acquired from the input image has a low reliability, to correct the white balance control signal by use of a white balance signal which was acquired before and which has been judged to have a high reliability, to provide it as output to the white balance means. According to this aspect, when the acquired white balance control signal has a low reliability, the correction is made by use of past white balance control signals having a high reliability, with the result that it is possible to achieve a good white balance to a certain extent even in the case of difficult scenes in which less reliable white balance control signals are only acquired.